Understanding and Supporting Debugging Workflows in Multiverse Analysis

Multiverse analysis-a paradigm for statistical analysis that considers all combinations of reasonable analysis choices in parallel-promises to improve transparency and reproducibility. Although recent tools help analysts specify multiverse analyses, they remain difficult to use in practice. In this work, we conduct a formative study with four multiverse researchers, which identifies debugging as a key barrier. We find debugging is challenging because of the latency between running analyses and detecting bugs, and the scale of metadata needed to be processed to diagnose a bug. To address these challenges, we prototype a command-line interface tool, Multiverse Debugger, which helps diagnose bugs in the multiverse and propagate fixes. In a second, focused study (n=13), we use Multiverse Debugger as a probe to develop a model of debugging workflows and identify challenges, including the difficulty in understanding the composition of a multiverse. We conclude with design implications for future multiverse analysis authoring systems

IIFL: Implicit Interactive Fleet Learning from Heterogeneous Human Supervisors

Imitation learning has been applied to a range of robotic tasks, but can struggle when (1) robots encounter edge cases that are not represented in the training data (distribution shift) or (2) the human demonstrations are heterogeneous: taking different paths around an obstacle, for instance (multimodality). Interactive fleet learning (IFL) mitigates distribution shift by allowing robots to access remote human teleoperators during task execution and learn from them over time, but is not equipped to handle multimodality. Recent work proposes Implicit Behavior Cloning (IBC), which is able to represent multimodal demonstrations using energy-based models (EBMs). In this work, we propose addressing both multimodality and distribution shift with Implicit Interactive Fleet Learning (IIFL), the first extension of implicit policies to interactive imitation learning (including the single-robot, single-human setting). IIFL quantifies uncertainty using a novel application of Jeffreys divergence to EBMs. While IIFL is more computationally expensive than explicit methods, results suggest that IIFL achieves 4.5x higher return on human effort in simulation experiments and an 80% higher success rate in a physical block pushing task over (Explicit) IFL, IBC, and other baselines when human supervision is heterogeneous

How Do Analysts Understand and Verify AI-Assisted Data Analyses?

Data analysis is challenging as it requires synthesizing domain knowledge, statistical expertise, and programming skills. Assistants powered by large language models (LLMs), such as ChatGPT, can assist analysts by translating natural language instructions into code. However, AI-assistant responses and analysis code can be misaligned with the analyst's intent or be seemingly correct but lead to incorrect conclusions Therefore, validating AI assistance is crucial and challenging. Here, we explore how analysts across a range of backgrounds and expertise understand and verify the correctness of AI-generated analyses. We develop a design probe that allows analysts to pursue diverse verification workflows using natural language explanations, code, visualizations, inspecting data tables, and performing common data operations. Through a qualitative user study (n=22) using this probe, we uncover common patterns of verification workflows influenced by analysts' programming, analysis, and AI backgrounds. Additionally, we highlight open challenges and opportunities for improving future AI analysis assistant experiences

Structural‐Deformation‐Energy‐Modulation Strategy in a Soft Porous Coordination Polymer with an Interpenetrated Framework

German version: https://doi.org/10.1002/ange.202003186To achieve unique molecular‐recognition patterns, a rational control of the flexibility of porous coordination polymers (PCPs) is highly sought, but it remains elusive. From a thermodynamic perspective, the competitive relationship between the structural deformation energy (Edef) of soft PCPs and the guest interaction is key for selective a guest‐triggered structural‐transformation behavior. Therefore, it is vital to investigate and control Edef to regulate this competition for flexibility control. Driven by these theoretical insights, we demonstrate an Edef‐modulation strategy via encoding inter‐framework hydrogen bonds into a soft PCP with an interpenetrated structure. As a proof of this concept, the enhanced Edef of PCP enables a selective gate‐opening behavior toward CHCl₃ over CH₂Cl₂ by changing the adsorption‐energy landscape of the compounds. This study provides a new direction for the design of functional soft porous materials

Scattering From a Two Dimensional Array of Flux Tubes: A Study of The Validity of Mean Field Theory

Mean Field Theory has been extensively used in the study of systems of anyons in two spatial dimensions. In this paper we study the physical grounds for the validity of this approximation by considering the Quantum Mechanical scattering of a charged particle from a two dimensional array of magnetic flux tubes. The flux tubes are arranged on a regular lattice which is infinitely long in the ``$y$'' direction but which has a (small) finite number of columns in the ``$x$'' direction. Their physical size is assumed to be infinitesimally small. We develop a method for computing the scattering angle as well as the reflection and transmission coefficients to lowest order in the Aharonov--Bohm interaction. The results of our calculation are compared to the scattering of the same particle from a region of constant magnetic field whose magnitude is equal to the mean field of all the flux tubes. For an incident plane wave, the Mean Field approximation is shown to be valid provided the flux in each tube is much less than a single flux quantum. This is precisely the regime in which Mean Field Theory for anyons is expected to be valid. When the flux per tube becomes of order 1, Mean Field Theory is no longer valid.Comment: 23 pages, University of British Columbia Preprint UBCTP93-01